Problem Note 45818: Slow performance when computing robust estimators of scale
A noticeable performance degradation might be observed on 64-bit operating systems other than Windows as compared to Windows operating systems when robust estimators of scale are requested in PROC UNIVARIATE. The robust estimators of scale are Gini's Mean Difference, the Interquartile Range, MAD, Qn, and Sn. When any of one of these estimators or its standard deviation is requested, they are all computed by the procedure internally. These statistics are requested in either of the following ways:
- Specifying the ROBUSTSCALE option in the PROC UNIVARIATE statement
- Specifying any of the following keywords on an INSET or OUTPUT statement:
- GINI
- MAD
- QN
- SN
- STD_GINI
- STD_MAD
- STD_QN
- STD_QRANGE
- STD_SN
It is the Qn computation that takes the longest and causes very slow performance. When the robust estimators are specified, there is an internal check that omits the computation of Qn and sets it to a missing value when the number of observations exceeds a preset value. On the Windows operating system, Qn is not computed when there are more than approximately 65,530 observations. However, on 64-bit operating systems other than Windows, it is not until the number of observations exceeds approximately 134,200,000 that the Qn computation is suppressed. Therefore, the Qn computation is carried out for much larger data sets on these 64-bit machines.
A Hot Fix is available that eliminates the computation of QN and STD_QN in cases where they are not specifically requested. The fix improves performance when any of GINI, MAD, SN, STD_GINI, STD_MAD, STD_QRANGE and STD_SN are requested, and QN and STD_QN are not requested. With the Hot Fix installed, only the requested robust estimators of scale are computed.
The Hot Fix causes no change in the performance of PROC UNIVARIATE if no robust estimators of scale are requested. There is also no change in the performance when QN or STD_QN are requested.
Operating System and Release Information
SAS System | Base SAS | Microsoft Windows Server 2003 Enterprise 64-bit Edition | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft Windows XP 64-bit Edition | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft Windows XP Professional | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft Windows Server 2008 for x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft® Windows® for 64-Bit Itanium-based Systems | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft Windows Server 2003 for x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft Windows Server 2003 Standard Edition | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft Windows Server 2003 Enterprise Edition | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft Windows Server 2003 Datacenter Edition | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft® Windows® for x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Microsoft Windows Server 2003 Datacenter 64-bit Edition | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Solaris for x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Linux for x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
64-bit Enabled Solaris | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Windows Vista for x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Windows 7 Ultimate x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Windows 7 Professional x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Windows 7 Home Premium x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
64-bit Enabled AIX | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
Windows 7 Enterprise x64 | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
64-bit Enabled HP-UX | 9.21_M3 | 9.3_M2 | 9.2 TS2M3 | 9.3 TS1M2 |
*
For software releases that are not yet generally available, the Fixed
Release is the software release in which the problem is planned to be
fixed.
Type: | Problem Note |
Priority: | medium |
Topic: | Analytics ==> Descriptive Statistics Analytics ==> Distribution Analysis Analytics ==> Exploratory Data Analysis SAS Reference ==> Procedures ==> UNIVARIATE
|
Date Modified: | 2016-01-08 10:45:29 |
Date Created: | 2012-02-27 14:42:28 |